An investigation of a tutor evaluation scale for formative purposes in a problem-based learning curriculum.

Tins arllcle wa,s accepledfOr publicalion janufll]' 25, J996. Problem-based learning (PBL) curricula are of increasing interest to educators within occupational therapy and other allied health professions (Bruhn, 1992; Hay, 1995; Jacobs & Lyons, 1992; Royeen, 1995; Saarinen & Salvatori, 1994; VanLeit, 1995), PBL approaches involve small groups of students who meet with a tutor to examine health care problems that highlight learning needs, With the tutor's guidance, these students develop learning objectives and establish strategies to meet them. The knowledge gained through this process appears to be well retained because the learning occurs in context and is responsive to student needs (Norman & Schmidt, 1992), Tutors are often skilled clinicians who are interested in education and are willing and able to become involved in PBL curricula, To provide feedback to tutors about which aspects of their performance need improvement, reliable evaluation tools are required. Teaching evaluation tools commonly used in traditional curricula are inappropriate for tutor evaluation because they measure didactic teaching skills rather than the facilitation and guidance skills required for student-directed learning in small groups (Irby, 1984; Rippey, 1981). The roles, requisite skills, and attitudes of tutors involved in student-directed learning have been well described (Barrows & Tamblyn, 1980; Knowles, 1975; Maxwell & Wilkerson, 1990; Schmidt & Moust, 1995). These attributes include the ability to facilitate the group process, communicate clearly, enable individual learning objectives, and provide constructive feedback. Each attribute can be evaluated by the student on the basis of their experience with the tutor as well as through direct observation, videotaping, or other more demanding means. The area of tutor evaluation has been largely unaddressed in the literature (Wakefield, Keane, & Hay, 1992). Additionally, recent investigations of tutor effectiveness that rely heavily on student evaluations provide few data on the psychometric properties of the evaluation tools (Schmidt & Moust, 1995), On the basis of a review of existing measures and discussions with experienced tutors, Hay (1996) and Hay and Schmuck (1993) developed a 19-item scale specifically for evaluation of tutors, Eighteen of the items address specific performance aspects (e.g" the provision of verbal feedback). The final item provides a global rating (see Figure 1). The scale has good interobserver and test-retest reliabilities. Fourteen items had test-retest (over a 5-day interval) reliabilities of more than ,70, and one had a reliability of less than ,50. Seven items had